4 research outputs found
Re-IQA: Unsupervised Learning for Image Quality Assessment in the Wild
Automatic Perceptual Image Quality Assessment is a challenging problem that
impacts billions of internet, and social media users daily. To advance research
in this field, we propose a Mixture of Experts approach to train two separate
encoders to learn high-level content and low-level image quality features in an
unsupervised setting. The unique novelty of our approach is its ability to
generate low-level representations of image quality that are complementary to
high-level features representing image content. We refer to the framework used
to train the two encoders as Re-IQA. For Image Quality Assessment in the Wild,
we deploy the complementary low and high-level image representations obtained
from the Re-IQA framework to train a linear regression model, which is used to
map the image representations to the ground truth quality scores, refer Figure
1. Our method achieves state-of-the-art performance on multiple large-scale
image quality assessment databases containing both real and synthetic
distortions, demonstrating how deep neural networks can be trained in an
unsupervised setting to produce perceptually relevant representations. We
conclude from our experiments that the low and high-level features obtained are
indeed complementary and positively impact the performance of the linear
regressor. A public release of all the codes associated with this work will be
made available on GitHub.Comment: Accepted to IEEE/CVF CVPR 2023. Code will be released post conference
in July 2023. Avinab Saha & Sandeep Mishra contributed equally to this wor
GAMIVAL: Video Quality Prediction on Mobile Cloud Gaming Content
The mobile cloud gaming industry has been rapidly growing over the last
decade. When streaming gaming videos are transmitted to customers' client
devices from cloud servers, algorithms that can monitor distorted video quality
without having any reference video available are desirable tools. However,
creating No-Reference Video Quality Assessment (NR VQA) models that can
accurately predict the quality of streaming gaming videos rendered by computer
graphics engines is a challenging problem, since gaming content generally
differs statistically from naturalistic videos, often lacks detail, and
contains many smooth regions. Until recently, the problem has been further
complicated by the lack of adequate subjective quality databases of mobile
gaming content. We have created a new gaming-specific NR VQA model called the
Gaming Video Quality Evaluator (GAMIVAL), which combines and leverages the
advantages of spatial and temporal gaming distorted scene statistics models, a
neural noise model, and deep semantic features. Using a support vector
regression (SVR) as a regressor, GAMIVAL achieves superior performance on the
new LIVE-Meta Mobile Cloud Gaming (LIVE-Meta MCG) video quality database.Comment: Accepted to IEEE SPL 2023. The implementation of GAMIVAL has been
made available online: https://github.com/lskdream/GAMIVA
Study of Subjective and Objective Quality Assessment of Mobile Cloud Gaming Videos
We present the outcomes of a recent large-scale subjective study of Mobile
Cloud Gaming Video Quality Assessment (MCG-VQA) on a diverse set of gaming
videos. Rapid advancements in cloud services, faster video encoding
technologies, and increased access to high-speed, low-latency wireless internet
have all contributed to the exponential growth of the Mobile Cloud Gaming
industry. Consequently, the development of methods to assess the quality of
real-time video feeds to end-users of cloud gaming platforms has become
increasingly important. However, due to the lack of a large-scale public Mobile
Cloud Gaming Video dataset containing a diverse set of distorted videos with
corresponding subjective scores, there has been limited work on the development
of MCG-VQA models. Towards accelerating progress towards these goals, we
created a new dataset, named the LIVE-Meta Mobile Cloud Gaming (LIVE-Meta-MCG)
video quality database, composed of 600 landscape and portrait gaming videos,
on which we collected 14,400 subjective quality ratings from an in-lab
subjective study. Additionally, to demonstrate the usefulness of the new
resource, we benchmarked multiple state-of-the-art VQA algorithms on the
database. The new database will be made publicly available on our website:
\url{https://live.ece.utexas.edu/research/LIVE-Meta-Mobile-Cloud-Gaming/index.html}Comment: Accepted to IEEE Transactions on Image Processing, 2023. The database
will be publicly available by 1st week of July 202
Perceptual video quality assessment: the journey continues!
Perceptual Video Quality Assessment (VQA) is one of the most fundamental and challenging problems in the field of Video Engineering. Along with video compression, it has become one of two dominant theoretical and algorithmic technologies in television streaming and social media. Over the last 2 decades, the volume of video traffic over the internet has grown exponentially, powered by rapid advancements in cloud services, faster video compression technologies, and increased access to high-speed, low-latency wireless internet connectivity. This has given rise to issues related to delivering extraordinary volumes of picture and video data to an increasingly sophisticated and demanding global audience. Consequently, developing algorithms to measure the quality of pictures and videos as perceived by humans has become increasingly critical since these algorithms can be used to perceptually optimize trade-offs between quality and bandwidth consumption. VQA models have evolved from algorithms developed for generic 2D videos to specialized algorithms explicitly designed for on-demand video streaming, user-generated content (UGC), virtual and augmented reality (VR and AR), cloud gaming, high dynamic range (HDR), and high frame rate (HFR) scenarios. Along the way, we also describe the advancement in algorithm design, beginning with traditional hand-crafted feature-based methods and finishing with current deep-learning models powering accurate VQA algorithms. We also discuss the evolution of Subjective Video Quality databases containing videos and human-annotated quality scores, which are the necessary tools to create, test, compare, and benchmark VQA algorithms. To finish, we discuss emerging trends in VQA algorithm design and general perspectives on the evolution of Video Quality Assessment in the foreseeable future